Computers operate under the control of a program comprising coded, executable instructions. Typically, a program is first written as a textual representation of computer-executable instructions in a high-level language, such as BASIC, C, C++, C#, or the like, which are more readily understood by humans. A file containing a program in high-level language form is known as source code. The high-level language statements of the source code are then translated or compiled into the coded instructions executable by the computer. Typically, a software program known as a compiler is used for this purpose.
Typically, the source code of a programming language is formed of program constructs organized in one or more program units, such as procedures, functions, blocks, modules, projects, packages and/or programs. These program units allow larger program tasks to be broken down into smaller units or groups of instructions. High-level languages generally have a precise syntax or grammar, which defines certain permitted structures for statements in the language and their meaning.
A compiler is a computer program that translates source code into another language, such as object code executable by a computer or an intermediate language that requires further compilation to be executable. Typically, a compiler includes several functional parts. For example, a conventional compiler may include a lexical analyzer that separates the source code into various lexical structures of the programming language, known as tokens, such as may include keywords, identifiers, operator symbols, punctuation, and the like.
Compilers are generally implemented as closed, non-extendable systems. A typical means to extend a compiler is to redesign, or revise, the compiler and/or implement pre-processing steps to translate language extensions such as additional data structures into units recognizable to the compiler. However, compiler design revisioning and/or employing pre-processors are both limited by the data structures inherent to the compiler.